Almathera Ten Pack 3: CDPD 3/Almathera Ten on Ten - Disc 3: CDPD3.iso/fish/001-100/001-025/014/dex/dex.ms

Text File | 1995-03-17 | 15KB | 720 lines

.ND June 28, 1983
.nr LL 6.5i
.nr PO 0.75i
.RP

.TL
DEX - Documentation Extraction Utility

.AU
Fred Fish

.AI
345 Scottsdale Road
Pleasant Hill, Ca  94523

.AB

.PP
Dex
is a utility for extracting documentation from program
source files and compiling it into a form suitable for input
to a text formatting program such as the UNIX "nroff" utility.

.PP
The primary benefits of a utility like
dex
are:
(1) internal source code documentation is far more
likely to remain current than external documentation,
(2) the production of updated documentation can be done
largely under computer control.

.PP
Dex
was developed by the author to aid in maintaining
documentation for his portable math library (pml).
Thus it is particularly useful in similar applications where
a programming project's source code is split into many small
files.
.AE

.bp
.NH 1
INTRODUCTION

.PP
Dex
is a utility for extracting documentation from program
source files and compiling it into a form suitable for
input to a text formatting program.  Currently, the only
text formatting program supported is the UNIX^
.FS
^UNIX is a trademark of Bell Laboratories.
.FE
"nroff" utility.
However, it would be a relatively simple process to extend
the current implementation to support other text processors.

.PP
Dex
was originally developed to aid the author in
maintaining his portable math library (pml) documentation.
However, dex
is useful for virtually any application where a medium to
large program is under development, particularly if the work
is being done simultaneously by several programmers.

.PP
Much of the
dex
implementation is done using
.ul
lex
and
.ul
yacc.
This has greatly reduced the amount of programming effort
required and made constant evolution relatively painless.
It does however effectively restrict usage of
dex
to UNIX
systems since few non-UNIX systems support these development
aids.

.bp
.NH 1
SOURCE CODE EXAMPLE

.NH 2
C Source Code Sample

.PP
The source code format for
dex
usage is quite similar to the
documentation format found in the Unix Programmer's Manual.
This leads to highly readable internal documentation in a familiar
format.
For example, the documentation shown in figure 1 might
precede the source code for a "tolower" library function.
.KF
.sp 3
.nf
   /*
    *  FUNCTION
    *
    *       tolower    convert character to lower case
    *
    *  SYNOPSIS
    *
    *       char tolower(ch)
    *       char ch;
    *
    *  DESCRIPTION
    *
    *       Tolower converts a character from upper case
    *       to lower case. Characters which are already
    *       lower case or are not alphabetic are returned
    *       unmodified.
    *
    *              a   =>   a
    *              A   =>   a
    *              &   =>   &
    *              7   =>   7
    *                  .
    *                 etc
    *
    */
.fi
.sp 2
.ce 1
Figure 1
.ce 1
.ul
Source Code Example
.sp 2
.KE

.NH 2
Dex Sample Output

.PP
Dex would process the text shown in figure 1, a documentation
.ul
region,
to produce the text formatter source in figure 2.
However, before you try it, at least read the section on dynamic
reconfiguration, since dex will do
.ul
nothing
without the appropriate reconfiguration file.

.KF
.nf
.sp 2
    .in 0
    .fi
    .bp
    .sp 3
    .ce
    ***********
    .ce
    * tolower *
    .ce
    ***********
    .sp 3
    .ul 1
    SYNOPSIS
    .sp 1
    .in 8
    .nf
    char tolower(ch)
    char ch;
    .sp 1
    .in 0
    .ul 1
    DESCRIPTION
    .sp 1
    .in 8
    .fi
    Tolower converts a character from upper case
    to lower case. Characters which are already
    lower case or are not alphabetic are returned
    unmodified.
    .sp 1
    .nf
    .in 16
    a   =>   a
    A   =>   a
    &   =>   &
    7   =>   7
        .
       etc
    .sp 1
.fi
.sp 2
.ce 1
Figure 2
.ce 1
.ul
Dex Sample Output
.sp 2
.KE

.NH 2
Nroff Sample Output

.PP
Dex currently knows nothing about macro packages, so its output
can be intermixed with other nroff source intended for any of the
macro packages  (with restrictions noted in macro package's documentation
concerning command conflicts).
Figure 3 shows the output of nroff for the input of figure 2.

.KF
.sp 2
.nf
                           ***********
                           * tolower *
                           ***********



.ul
SYNOPSIS

        char tolower(ch)
        char ch;

.ul
DESCRIPTION

        Tolower converts a character from  upper  case  to  lower
        case.  Characters which are already lower case or are not
        alphabetic are returned unmodified.

                a   =>   a
                A   =>   a
                &   =>   &
                7   =>   7
                    .
                   etc
.fi
.sp 2
.ce 1
Figure 3
.ce 1
.ul
Nroff Sample Output
.sp 2
.in +4
.KE

.NH 2
Dynamic Reconfiguration

.PP
Dex has
.ul
no
built in knowledge about how to handle each documentation
.ul
topic
(such as "DESCRIPTION")
with respect to formatter modes such
as fill, underline, indent, etc.
This knowledge is obtained via a mechanism referred to as
.ul
dynamic reconfiguration,
whereby a special file is read from the directory containing
the processed source files.
Dex can also be instructed to read a reconfiguration file
of the user's choosing.

.PP
Dex does not complain about the absence of the reconfiguration
file and if it is unreadable will essentially do nothing while
processing the specified source files.
In the example of figure 1,
dex
would typically be reconfigured to default to fill
mode for the text of the "DESCRIPTION"
.ul
topic
and to default to nofill mode for the text "SYNOPSIS"
.ul
topic.

.bp
.NH 1
DEFINITIONS

.PP
To provide a consistent terminology for describing dex usage
some basic "buzz words" need to be defined.

.NH 2
Documentation Topic

.PP
The basic building block of internal documentation is the documentation
.ul
topic.
A documentation
.ul
topic
is a contiguous section of documentation which begins with recognition
of a topic
.ul 1
identifier
and continues until recognition of
another topic
.ul
identifier,
or program source code (junk).

.PP
In the previous example, the topic
.ul
identifiers
are "FUNCTION", "SYNOPSIS", "DESCRIPTION", and "BUGS".
The
.ul
topics
are the topic
.ul
identifiers
along with the corresponding topic
.ul
body
(text).
Thus the "BUGS"
.ul
topic
consists of the the lines shown in figure 4.

.KF
.sp 3
.nf
.in +4
 *  BUGS
 *
 *      As implemented only works on systems for which
 *      native character set is ASCII.
 *
.in -4
.fi
.sp 2
.ce
Figure 4
.ul
.ce
Documentation Topic Example
.sp 2
.KE

.NH 2
Documentation Region

.PP
A documentation
.ul 1
region
is a (possibly non-contiguous) section of documentation
comprised of one or more documentation
.ul
topics.
It begins with recognition of a topic
identifier which has previously been configured to start
a region and continues until another region start identifier
is encountered.

.PP
In the example of figure 1, only the topic identifier "FUNCTION"
will generally be flagged as starting a new documentation
region.
Thus the entire C comment is a documentation
region
although it contains no embedded C code between topics.

.bp
.NH 1
SOURCE LINE DETAILS

.PP
Dex is set up to recognize comment lines for various source file
types.  It recognizes the "*" character as starting a C comment
line, the "#" character as starting "make" and dex reconfiguration
file comment lines, and a ";" character or "|" character
as starting assembler
comment lines.  These characters can be preceeded by any number
of blanks and tabs and must be followed by at least one blank
or tab.

.PP
Files processed by dex are divided strings which match one of the following
forms:
.sp 2
.nr LL -0.5i
.in +2
.IP (1)
<blanks/tabs><comment_string><blanks><text><newline>
.bk
Usually contains the topic identifier in the "text" field.
The <blanks> field consists of 2 or more blanks.
.IP (2)
<blanks/tabs><comment_string><tab><text><newline>
.bk
The text is emitted in fill or nofill mode depending
upon the state of the EMITFILL flag (see reconfiguration
section).
.IP (3)
<blanks/tabs><comment_string><tab><tab><text><newline>
.bk
The text is emitted in nofill mode
.ul
regardless
of the state of the EMITFILL flag.
The <text> is anything except newline, including
blanks and tabs.
.IP (4)
.bk
<blanks/tabs><comment_string><newline>
Causes commands for a single blank line to be emitted.
Thus blank comment lines map one for one with blank lines in the
nroff output.
.IP (5)
Any line not beginning with <blanks/tabs> followed
by <comment_string> is
considered to be "junk" and terminates processing of the
current topic.  Subsequent lines which match (2), (3), or (4) above
will be ignored until the next match of (1).
.in -2
.nr LL +0.5i

.PP
Figure 1 is reproduced in figure 5 with each line marked
according to which pattern it matches.  Carefully study
figures 1, 2, 3, and 5 to determine what transformations
are made in going from the source code to the nroff output.
.KF
.sp 2
.nf
    5   /*
    1    *  FUNCTION
    4    *
    2    *       tolower    convert character to lower case
    4    *
    1    *  SYNOPSIS
    4    *
    2    *       char tolower(ch)
    2    *       char ch;
    4    *
    1    *  DESCRIPTION
    4    *
    2    *       Tolower converts a character from upper case
    2    *       to lower case. Characters which are already
    2    *       lower case or are not alphabetic are returned
    2    *       unmodified.
    4    *
    3    *              a   =>   a
    3    *              A   =>   a
    3    *              &   =>   &
    3    *              7   =>   7
    3    *                  .
    3    *                 etc
    4    *
    5    */
.fi
.sp 2
.ce 1
Figure 5
.ce 1
.ul
Source Code Example Revisited
.sp 2
.KE
.PP
Note that there can be any number of leading tabs and
blanks (or none).  Also, lines which have exactly one
tab after the string recognized as starting a comment
can be emitted in either filled or unfilled mode while
lines having two tabs are emitted in unfilled mode
only.  This allows reasonable handling and appearance
of fillable text interspersed with lists or other
non-fillable text.

.PP
It should also be pointed out that the file being processed
does
.ul
not
have to be an ASCII text file.  All input bytes are masked
with octal 177 so object files and other non-ASCII files
do not cause problems.  This allows one to scan all
files in a directory, extracting documentation, without
worrying about what kind of files they are.


.bp
.NH 1
RECONFIGURATION

.PP
Dex is designed so that it can be reconfigured
dynamically by placing reconfiguration
instructions in a file in the same directory as the files to
be processed.
If a file name other than the default
reconfiguration file is desired, this too can be handled by issuing
the appropriate command line switch when
dex
is invoked.

.PP
Each time
dex
processes the first file in any directory it
automagically looks for a file called ".dexrc", the
default dex reconfiguration file.
It will
.ul
not
complain if the file is not found or is unreadable.
Some of the things which can be reconfigured are:
.sp 1
.nr LL -0.5i
.in +4
.IP o
.ul
Topic identifier words:
Specific strings can be added to or removed from the internal
tables.
.IP o
.ul
Fill mode:
The default handling for fill/unfill mode can be enabled or disabled
for specific documentation topics.
.IP o
.ul
Output files:
The output for specific documentation regions can be
redirected to files other than the default standard output.
.IP o
.ul
Processing inhibition:
Processing of specified documentation regions or topics
can be disabled or enabled dynamically.
.nr LL +0.5i
.in -4
.sp 1

.NH 2
Reconfiguration File Format

.PP
Each line of a reconfiguration file is either a comment or a
reconfiguration directive.  Comments are any lines
beginning with the "#" character (with or without leading
whitespace), a blank, or a form feed character.
The directives currently supported are:
.sp 1
.nr LL -0.5i
.in +8
.ul
.IP \.flags 10
Set or reset flags for a given topic identifier.
.ul
.IP \.output 10
Redirect output to a file for a given topic identifier.
.nr LL +0.5i
.in -8
.sp 1

.PP
More than one directive line may be associated with any given
identifier.   Thus, for example, if one line is insufficient to set or
reset all desired flags for the "FILE" identifier, then another
directive can be issued for the remaining flags.

.PP
Entries are free format within a given line; whitespace separating
directive fields is simply ignored.  The formats of the current
directives are:
.sp 2
.nf
        .flags  <identifier>  [-]<flag1> [-]<flag2> ...

        .output <identifier>  <filename>
.fi
.sp 2

.NH 2
Identifier Flags

.PP
Flags associated with specified topic identifiers can be set or
reset by the ".flags" reconfiguration directive.  When dex starts
up all flags are initially reset.
They are set by simply naming them in the directive line (a leading
"+" character is optional).
They are reset by naming them with a leading "-" character.  Flags
not named in the reconfiguration file remain unchanged.
The current flags are:
.sp 2
.nr LL -0.5i
.in +8
.IP PROCESS 12
If set then topic is processed, otherwise it is ignored.
.IP EMITTEXT 12
If set then topic text is emitted, otherwise there is not
output while topic is processed.
.IP EMITBOX 12
If set then first word of topic text is emitted enclosed
in a box.  (Doesn't mesh well with EMITTEXT)
.IP EMITFILL 12
If set then text separated from comment string ("*", "#", or
";") with
a single tab is emitted in fill mode.
.IP EMITUL 12
If set then topic identifier is underlined when emitted.
.IP EMITBP 12
If set then commands to start new page are emitted prior
to topic output (generally used with REGION).
.IP REGION 12
If set then topic identifier is considered to start a new
documentation region.  If PROCESS is simultaneously reset
then
.ul
all
topics until the next region are suppressed.
.nr LL +0.5i
.in -8
.sp 2

.NH 2
Reconfiguration File Example

.PP
Figure 6 is a reconfiguration file which specifies that the identifier
"FUNCTION" starts a documentation region, the region is to
be processed, the function name is to be printed in a box and
each function starts on a new page.   Also, the identifier "DESCRIPTION"
is for a normal (non-region) topic, the topic is processed, its
text is emitted, the emitted text is fill mode by default, and
the identifier is to be underlined when printed.
.KF
.nf
.sp 3
      .flags "FUNCTION"     REGION PROCESS EMITBOX EMITBP
      .flags "DESCRIPTION" -REGION PROCESS EMITTEXT
      .flags "DESCRIPTION"  EMITUL EMITFILL
.sp 2
.ce
Figure 6
.ul
.ce
Reconfiguration File Example
.fi
.sp 2
.KE

.bp
.NH 1
MISCELLANEOUS

.NH 2
Usage

.PP
Dex is invoked with a command line of the form:
.sp 2
.nf
    dex [-dhtv] [-r rcfile] file1 file2 file3 ...

        d =>  enable debug mode
        h =>  print internal help message
        t =>  enable trace mode
        v =>  enable verbose mode

        r =>  use reconfiguration file <rcfile>
.fi
.sp 1

.NH 2
Style Notes

.PP
The following order is suggested for topics within a documentation region.
It roughly follows the order which appears in the Unix Programmer's Manual.

.sp 1
.in +4
.IP o
FUNCTION or FILE or TOOL or NAME
.IP o
KEY WORDS
.IP o
SYNOPSIS
.IP o
DESCRIPTION
.IP o
RETURNS
.IP o
EXAMPLE
.IP o
FILES
.IP o
SEE ALSO
.IP o
DIAGNOSTICS
.IP o
WARNINGS or CAVEATS or RESTRICTIONS or BUGS
.IP o
AUTHOR
.IP o
PSEUDO CODE
.in -4
.sp 2